Machine Learning at the Edge

EMB 345: Machine Learning at the Edge focuses on the critical aspects of integrating artificial intelligence into embedded systems. Students will gain a deep understanding of the challenges and opportunities associated with running machine learning inference on devices with limited computational power, memory, and energy. The course covers the entire lifecycle of edge AI development, from model selection and optimization to deployment and monitoring on target hardware. Key topics include model quantization, pruning, neural network architectures suitable for edge devices, hardware acceleration techniques, and ethical considerations. Practical laboratory exercises will emphasize hands-on experience with popular embedded platforms and toolchains used for deploying ML models. By the end of this course, students will be equipped to design, implement, and evaluate efficient machine learning solutions for various edge computing applications.

Prerequisites

EMB 315

Course outline

Lectures, virtual labs, and graded assignments — completed in your browser.

01Introduction to Edge ML: Concepts and Challengeslecture

02Edge AI Hardware Landscape: MCUs, NPUs, and Acceleratorslecture

03Assignment 1: Edge AI Platform Research and Selectionassignment

04Model Optimization: Quantization and Pruning Techniqueslecture

05Lab 1: Quantizing a TensorFlow Lite Modellab

06Efficient Neural Network Architectures for Edgelecture

07Deploying Models with TensorFlow Lite Microlecture

08Lab 2: Deploying an Image Classifier on an MCUlab

09Assignment 2: Edge ML Application Design Proposalassignment

10Data Preprocessing and Sensor Integration at the Edgelecture

11Edge AI Security, Privacy, and Ethical Considerationslecture

12Lab 3: Real-time Object Detection on an Edge Devicelab

13Comprehensive Midterm Reviewlecture

14Final Reviewlecture

Syllabus

## EMB 345: Machine Learning at the Edge

### Course Outcomes
Upon successful completion of this course, students will be able to:
* CO1: Explain the fundamental concepts and challenges of deploying machine learning models on embedded systems.
* CO2: Select and optimize appropriate machine learning models for resource-constrained edge devices.
* CO3: Apply techniques such as quantization, pruning, and neural architecture search to reduce model complexity and size.
* CO4: Implement and evaluate machine learning inference on various embedded hardware platforms.
* CO5: Analyze the trade-offs between model accuracy, latency, power consumption, and memory footprint in edge AI applications.
* CO6: Design and develop end-to-end edge AI solutions for practical problems.

### Weekly Topics
1. Introduction to Machine Learning at the Edge: Concepts, Challenges, and Opportunities
2. Overview of Edge AI Hardware: Microcontrollers, NPUs, GPUs for Edge
3. Fundamentals of Machine Learning for Embedded Systems
4. Model Quantization: Principles and Techniques
5. Model Pruning and Sparsity for Edge Deployment
6. Efficient Neural Network Architectures for Edge Devices (MobileNet, SqueezeNet, etc.)
7. Hardware Acceleration and Specialized Edge AI Processors
8. Deep Dive into TensorFlow Lite and ONNX Runtime
9. Lab: Deploying a Simple Classifier on an MCU
10. Data Pipelines and Preprocessing for Edge ML
11. Edge AI Security, Privacy, and Ethical Considerations
12. Project Development: Designing an Edge AI Application
13. Comprehensive Midterm Review
14. Final Review

### Grading Policy
* Knowledge Checks: 15%
* Assignments/Labs: 30%
* Quizzes: 25%
* Final Exam: 30%

### Required Materials
* Textbook: No single textbook is required. Readings will be provided via the LMS.
* Hardware: ESP32 development board, Raspberry Pi (or similar), camera module.
* Software: Python 3, TensorFlow Lite, PyTorch Mobile, selected IDEs (e.g., VS Code with PlatformIO, Jupyter).